A smart method for spark using neural network for big data
نویسندگان
چکیده
Apache spark, famously known for big data handling ability, is a distributed open-source framework that utilizes the idea of memory to process data. As performance spark mostly being affected by predominant configuration parameters, it challenging achieve optimal result from spark. The current practice tuning parameters ineffective, as performed manually. Manual large space and complex interactions with among parameters. This paper proposes more effective, self-tuning approach subject neural network called Smart method using (SSNNB) avoid disadvantages manual has selected five different sizes test approach. proposed increased speed around 30% compared default parameter configuration.
منابع مشابه
a new approach to credibility premium for zero-inflated poisson models for panel data
هدف اصلی از این تحقیق به دست آوردن و مقایسه حق بیمه باورمندی در مدل های شمارشی گزارش نشده برای داده های طولی می باشد. در این تحقیق حق بیمه های پبش گویی بر اساس توابع ضرر مربع خطا و نمایی محاسبه شده و با هم مقایسه می شود. تمایل به گرفتن پاداش و جایزه یکی از دلایل مهم برای گزارش ندادن تصادفات می باشد و افراد برای استفاده از تخفیف اغلب از گزارش تصادفات با هزینه پائین خودداری می کنند، در این تحقیق ...
15 صفحه اولForecasting Natural Gas Demand Using Meteorological Data: Neural Network Method
The need for prediction and patterns of gas consumption especially in the cold seasons is essential for consumption management and policy planning decision making. In residential and commercial uses which account for the bulk of gas consumption in the country the effects of meteorological variables have the highest impact on consumption. In the present research four variables include daily ave...
متن کاملA conjugate gradient based method for Decision Neural Network training
Decision Neural Network is a new approach for solving multi-objective decision-making problems based on artificial neural networks. Using inaccurate evaluation data, network training has improved and the number of educational data sets has decreased. The available training method is based on the gradient decent method (BP). One of its limitations is related to its convergence speed. Therefore,...
متن کاملA New Method for Intrusion Detection Using Genetic Algorithm and Neural Network
The article attempts to have neural network and genetic algorithm techniques present a model for classification on dataset. The goal is design model can the subject acted a firewall in network and this model with compound optimized algorithms create reliability and accuracy and reduce error rate couse of this is article use feedback neural network and compared to previous methods increase a...
متن کاملA New Method for Intrusion Detection Using Genetic Algorithm and Neural Network
The article attempts to have neural network and genetic algorithm techniques present a model for classification on dataset. The goal is design model can the subject acted a firewall in network and this model with compound optimized algorithms create reliability and accuracy and reduce error rate couse of this is article use feedback neural network and compared to previous methods increase a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: International Journal of Power Electronics and Drive Systems
سال: 2021
ISSN: ['2722-2578', '2722-256X']
DOI: https://doi.org/10.11591/ijece.v11i3.pp2525-2534